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ABSTRACT 

This  report  contains  the  ideas  shared  during  our  syndicate  sessions.  We  tried  to  use  these  few  hours  of 
brainstorming  to  come  up  with  guidelines  about  how  to  improve  future  visualization  techniques  to 
address  military  issues.  The  results  of  our  discussions  consist  of  a)  advice  on  how  the  industry, 
the  research  community  and  user  representatives  should  work  together  to  best  serve  the  military  needs, 
b)  a  preliminary  look  at  how  sensors  and  weapons  capabilities,  performance  and  constraints  could 
be  presented,  c)  additional  areas  of  research  that  were  identified  while  considering  military  issues,  and 
d)  some  guidelines  about  how  to  evaluate  a  visualization  tool. 


1.0  ADVICE  ON  WORK  PROCESS 

Considering  the  urgency  and  the  complexity  of  the  military  needs  for  visualization  tools,  we  concluded 
that  both  trial-and-error  quick  prototypes  and  longer  term  research  to  solve  hard  problems  are  equally 
important.  Rigorous  research  is  obviously  necessary  to  tackle  the  many  hard  problems  in  presenting  just 
what  the  users  need  to  perform  theirs  task.  However,  systems  should  be  put  in  the  users’  hands  as  quickly 
as  possible.  Fast-prototyping  and  trial  and  error  are  appropriate  methods  for  this  task  since  the  users’  needs 
will  change  as  they  get  to  interact  with  the  tools.  Users  are  crucial  in  the  development  loop  since  someone 
has  to  be  very  knowledgeable  about  the  domain  to  know  what  elements  of  information  need  to  be 
presented.  A  related  fact  mentioned  was  that  a  lack  of  representative  data  often  ends  up  being  a  serious 
bottleneck  in  the  development  process  of  visualization  tools.  Hence,  regular  exchange  points  need  to  be 
scheduled  between  industry,  research  community  and  military  personnel  in  order  to  exchange  needs,  views 
and  results.  This  would  also  help  to  get  the  different  parties  to  talk  the  same  language  and  work  in  a  way 
that  would  minimize  redundancy  and  facilitate  integration. 


Paper  presented  at  the  RTO  1ST  Workshop  on  “ Massive  Military  Data  Fusion  and  Visualisation:  Users  Talk 
with  Developers”,  held  in  Halden,  Norway,  10-13  September  2002,  and  published  in  RTO-MP-105. 
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2.0  PROCESS  ILLUSTRATION  (How  We  Came  to  Choose  Our  Topic) 

The  keynotes  speakers  strongly  communicated  their  need  for  a  better  common  operational  picture  (COP). 
The  first  step  for  our  syndicate  was  then  to  define  which  elements  of  information  should  be  presented  to 
satisfy  this  need. 

Our  user  representative  produced  a  list  of  elements  of  interest.  An  attempt  at  mapping  visualization 
approaches  to  the  different  elements  of  information  in  the  domain  context  brought  to  the  foreground  the 
challenge  of  not  only  dealing  with  huge  amount  of  physical  information,  but  also  the  need  to  extract  much 
less  tangible  information,  such  as  ‘intentions’,  ‘uncertainties’,  ‘threats’,  ‘operational  readiness’,  etc. 

Another  point  that  was  emphasized  by  the  user  representatives  during  this  workshop  is  the  fact  that  in  the 
new  higher  tempo  kind  of  warfare,  higher-level  decision-makers  are  more  likely  to  be  faced  with  very 
little  time  to  make  very  difficult  decisions.  It  follows  that  to  be  useful  in  that  kind  of  critical  situation,  the 
COP  needs  to  provide  more  than  situation  awareness:  it  needs  to  become  more  of  a  decision  aid. 

The  challenge  consists  in  reducing  latency  in  the  decision  cycle  by  creating  visualizations  that  emphasize 
‘actionable’  information. 

Based  on  our  user  representative  area  of  expertise  and  knowledge  of  what  is  currently  poorly  done, 
we  narrowed  our  focus  to  look  at  possible  ways  to  visualize  sensor  and  weapon  capabilities  and 
constraints.  Furthermore,  we  decided  to  look  at  how  decision  points  and  timelines  could  be  extracted  from 
these  visualizations  based  on  movement  and  dynamics  of  the  situation. 


3.0  DOMAIN  CONTEXT  FOR  PRESENTATION  TECHNIQUES  NEEDS 

The  issue  of  choosing  the  right  visualization  approach  to  present  particular  elements  of  information  caught 
the  interest  of  this  syndicate. 
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The  work  of  the  TTCP  C3I  Action  Group  on  Information  Visualization  (AG-3)  presented  by  Denis  Gouin 
during  this  workshop  is  a  good  start  at  tackling  this  issue.  Part  of  that  effort  was  the  creation  of  a  toolkit 
which  queries  databases  to  find  the  visualization  tools  supporting  a  specific  analysis  task.  It  would  be 
beneficial  for  the  visualization  community  to  produce  or  gain  access  to  such  databases  in  order  to  facilitate 
the  creation  of  an  infrastructure/framework  for  the  domain. 

Our  initial  attempt  (far  from  complete)  to  map  visualization  techniques  to  the  elements  of  information 
needed  for  our  selected  topic  is  shown  in  the  figure  below.  The  visualization  approaches  listed  in  the 
diagram  were  taken  from  the  paper  Denis  Gouin  wrote  for  this  workshop. 


Sensor/ Weapons  Capabilities/Constraints 

Descriptive  Aspects  Visualization  Approaches 


Physical 

Range 

—  Power 

—  Environment  \ _ 

—  Weather 

—  Dust 

—  Land  Masses  . 

—  Temperature 

—  Salinity 

—  Depth 
Flight  Path 
Cutouts 

Vs.  What  target? 

Less  tangible 

(abstract,  non-spatial, 
relations,  processes) 

What  is  this  ship  intention? 

What  does  he  know? 

What  can  he  do? 

How  reliable  is  that  info? 

Is  it  a  threat? 

How  much  risk? 

When  is  the  latest  time  I  can  switch  my  sensors  on? 

If  need  to  effectuate  evasive  manoeuver,  where  can  I  go? 


Visual 

representation 


2D  maps 
3D  maps 


Graphs 
Table  /  spreadsheet 
Organization  charts 
3D  charts  (e.g.  volumes) 

Gantt  charts 
dashboards 
Link  analysis 
outline/tree  view 
video 

ticker  tape 
symbology  /  icons 

Visualization 

enhancements 


Overlays 


Distortion 


Animation 


Interaction 

Drill  down 
drag  and  drop 
haptic  techniques 

drawing 
sharing  views 

control  focus  (e.g.  Dim  map) 
expand/contract  tree  view 
nav  info  categories  (e.g.  Tabs) 
hyperlinks 

Deployment 

Display  devices 
knowledge  wall 
multiple  displays 
table 
laptop 
handheld 
3D  stereo 
immersive  VR 

COTS  software 
portal 

spreadsheet 


4.0  SELECTED  TOPIC  -  SENSORS  AND  WEAPONS  CAPABILITIES  / 
PERFORMANCE  /  CONSTRAINTS 

4.1  Present  sensors  and  weapons  (and  communications)  performance  constraints 
showing  3D  contours  based  on  varying  parameters  values  such  as: 

•  Environment  (weather,  terrain,  salinity) 

•  Emission  levels 

•  Probability  of  kill 

•  Countermeasures 

•  Time 

By  showing  the  sensors  and  weapons  volume  coverage  or  range,  what  is  not  covered  is  also  highlighted, 
which  might  be  even  more  crucial  in  certain  situations.  Discontinuation  of  target  tracking  can  then  be 
predicted.  Likewise,  in  a  sensor  network,  the  likely  start  of  track  reporting  by  own  (firing  quality)  sensors 
can  be  predicted. 
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4.2  Showing  target  objects  in  relation  to  volume  presentation  -  and  volumes  in  relation 
to  other  volumes 

As  Battlespace  volumes  (performance  constraints)  of  opposing  platforms  or  forces  approach  each  other, 
timelines  and  decision  points  can  be  extracted  from  presentation. 

The  view  should  allow  users  to  answer  questions  at  a  glance  like: 

•  What  are  the  margins  for  detection  of  own  platform/force  based  on  the  uncertainty  regarding  the 
constraints  of  enemy  sensors? 

•  What  are  the  margins  for  detection  of  enemy  platforms/forces  based  on  the  uncertainty  regarding 
the  constraints  of  own  sensors? 

•  When  is  the  likely  moment  of  first  enemy  hostile  act? 

•  How  reliable  is  the  info  about  the  other  ship’s  location  with  respect  to  the  constraints  of  my 
sensors? 

•  When  is  the  latest  time  I  can  switch  my  sensors  on  in  order  to  avoid  early  detection  and  to  employ 
own  weapons  at  their  maximum  capability? 

The  box  at  the  top  left  of  the  sketch  below  illustrate  the  idea  of  listing  possible  decisions  that  could  be 
analysed  visually  by  selecting  it.  For  example,  choosing  attack  could  overlay  on  the  picture  time,  risk,  etc. 
involved  in  that  action. 


4.3  Optimizing  Volume  Presentations 

Volumetric  presentations  would  convey  the  capabilities/constraints  well,  but  experimentation  is  needed  to 
be  able  to  show  complex  intersecting  volumes  in  a  clear  simple  way  as  well  as  targets  appearing  in  front 
of,  inside  of  or  behind  volumes. 


What  if  we  want  to: 

®  attack 

avoid  confrontation 
(Not  included  in  sketch) 


Satellite 


Cloud 
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5.0  OTHER  INTERESTING  AREAS  OF  RESEARCH 

5.1  Displaying  Levels  of  Abstraction  for  Various  Users 

Different  users  at  different  levels  of  the  military  organization  need  different  perspectives  or  level  of  details 
to  accomplish  their  respective  tasks.  Furthermore,  commanders  might  find  it  useful  at  times  to  look  at 
what  his  or  her  subordinates  see.  Ideally,  one  tool  would  allow  for  continuous  levels  of  abstraction. 

The  challenge  is  to  create  a  set  of  rules  that  define  how  the  elements  of  information  can  be  aggregated 
when  a  simplified  picture  is  needed. 

5.2  Provide  Alternate  Modalities 

One  noticeable  after  effect  of  September  1 1  attacks  is  that  the  amount  of  information  to  be  processed  has 
increased  dramatically.  The  information  comes  from  multiple  sources  and  in  different  forms  (voice,  chat, 
email,  message,  etc).  Fusion  needs  to  be  applied  to  eliminate  excess  and  redundancy,  but  without  slowing 
the  traffic  too  much  since  a  lot  of  that  information  might  be  time  sensitive. 

It  would  be  useful  to  find  ways  to  level  out  the  modalities  used  to  process  the  information.  The  simple 
example  brought  up  to  explain  the  idea  was  your  colleague  writing  ‘Lunch  time!’  on  a  piece  of  paper  and 
showing  it  to  you  while  you  are  talking  on  the  phone. 

5.3  How  “Real”  is  Necessary? 

Comments  from  the  user  representatives  such  as  “I  want  to  see  a  helicopter,  not  a  dot.”,  or  “Resolution  is 
never  good  enough.”  highlighted  the  question  “How  ‘Real’  is  necessary?”. 

Obviously,  it  depends  on  the  task.  Realistic  looking  3D  models  of  airplanes  might  be  attractive,  but  if, 
in  order  to  recognize  the  type  of  aircraft,  the  models  need  to  be  so  big  that  it  becomes  very  hard  to 
determine  their  altitude  or  relative  distance,  then  it  fails. 

In  this  case,  the  challenge  is  to  create  simple  icons  that  emphasize  the  crucial  differences  between  the 
aircrafts  that  are  used  to  tell  them  apart. 

On  the  other  hand,  it  was  raised  that  the  traditional  military  symbols  need  to  be  replaced  because  in  the 
context  of  coalitions,  language  and  cultural  differences  pose  a  serious  problem. 

SPAWAR  developed  the  so-called  ‘Symbicons’  to  deal  with  that  issue.  At  the  time  presented,  they  were 
rejected,  but  Col.  Johansen  suggested  that  they  should  try  again,  now  that  the  need  is  more  obvious. 

Generally  speaking,  it  would  be  useful  to  come  up  with  guidelines  defining  when  symbols/graphs  work 
better  than  realistic  representations. 

5.4  Automatic  Stress/Workload  Detection 

This  provocative  topic  deals  with  the  fact  that  under  stress,  humans  have  a  tendency  to  narrow  their  focus 
and  lose  important  context  or  big  picture.  The  effect  is  called  tunneling.  It  might  be  interesting  to 
investigate  if  a  system  could  automatically  detect  the  level  of  stress  and  the  cognitive  load  of  the  user  and 
when  those  passes  certain  thresholds,  the  system  would  adaptively  change  the  level  of  information  being 
presented,  forcing  the  user  to  pull  back  and  look  at  the  big  picture  again. 
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5.5  Automatic  Display  of  Threats 

This  is  another  provocative  suggestion  that  is  partly  based  on  Col.  Johansen  comment  about  the  annoyance 
caused  by  the  time  it  takes  to  put  on  the  screen  the  particular  view  that  is  needed  at  a  particular  moment. 
He  mentioned  that  when  things  are  planned  and  most  likely  outcomes  are  known,  the  staff  will  setup  the 
views  that  might  be  needed  in  advance.  Maybe  a  system  of  bookmarks  could  be  implemented  to  facilitate 
that  solution. 

With  the  assumptions  that  a  system  could  automatically  detect  threats  and  know  which  views  are  needed 
to  deal  with  them,  then  an  alert  could  appear.  Selecting  it  would  display  the  right  view  to  look  at  the  threat. 
Unless  the  threat  is  much  more  crucial  than  the  task  at  hand,  it  would  probably  be  better  to  let  the  operator 
decide  when  to  change  what  is  shown  on  display. 


6.0  EVALUATION/ASSESSMENT  ISSUES 

Finally,  our  syndicate  also  discussed  the  issue  of  assessing  visualization  systems.  There  is  a  need  for 
metrics  to  assess  its  value.  As  mentioned  above,  the  fact  that  The  user  likes  it’  might  not  necessarily  be 
productive.  Therefore  here  are  a  few  guidelines  that  should  be  considered: 

Go  beyond  the  HAT  report  model  - 

This  is  needed  because  “cognitive  walkthroughs”  and  comparable  methods  are  NOT  good  predictors 
of  usability  in  practice. 

Elaborate  on  what  the  system  is  used  for.  Develop  a  representative  sample  of  tasks. 

Develop/agree  upon  measures  of  task  performance,  process  as  well  as  ultimate  outcome. 

If  the  system  exists  -  Conduct  experimental  studies  of  users  performing  the  sample  tasks. 

A  reasonable  number  of  users  should  be  greater  than  ten. 

The  users  involved  in  the  experiment  should  get  appropriate  prior  training  before  performance 
measurements. 

If  the  system  does  not  yet  exist,  but  there  is  a  design  -  Apply  cognitive  modelling  to  the  sample  of 
tasks  (Card,  Moran  &  Newell,  Kieras  &  John). 

This  step  may  reveal  that  the  task  requirements  cannot  be  met. 

It  also  allows  comparisons  of  alternative  designs. 

However,  the  scope  of  what  can  be  modelled  today  is  somewhat  limited. 


7.0  CONCLUSIONS 

The  sole  fact  that  some  results  could  be  presented  in  this  report  should  highlight  the  success  of  the 
syndicate  concept  with  respect  to  having  members  of  the  different  communities  exchange  data  and  views. 
This  is  a  step  in  the  right  direction,  but  much  more  efforts  need  to  be  devoted  to  the  creation  of  a  solid 
infrastructure  to  support  the  creation  of  innovative  visualization  techniques  that  would  effectively  address 
the  issues  the  military  is  faced  with  now  and  will  be  facing  in  the  future. 
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SYMPOSIA  DISCUSSION  -  SYNDICATE  5 


Comment: 

Recommend  that  there  is  effort  and  resources  put  into  a  decision  tool  that  looks  at  resource  distribution. 

Comment: 

In  the  NATO  context,  the  political  context  is  important  to  the  common  operational  picture. 

Question: 

What  impact  can  the  three  different  domains  -  physical,  information,  cognitive  domain  (including  moral, 
ethical,  and  political)  -  have  on  development  of  decision  support  tools? 

Response: 

Decision  support  tools  can  be  applied  in  all  domains,  but  analysis  of  the  domains  and  requirements  is 
need.  Research  is  just  in  the  beginning  of  defining  those  requirements.  Focus  right  now  is  on  the  physical 
domain. 
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Sensors  and  weapons 
capabilities/performance/constraints 

•  Present  sensors  and  weapons  constraints  showing  3D 
contours  based  on  varying  parameters  values  such  as 

-  Environment  (weather,  terrain,  salinity) 

-  Probability  of  kill 

-  Countermeasures 

-  Time 

•  Showing  target  objects  in  relation  to  volume  presentation 
-  and  volumes  in  relation  to  other  volumes 

-  Answer  questions  at  a  glance 

-  ‘What  if  interactions 

•  Optimizing  volume  presentations 
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Volumetric  presentations 


What  if  we  want  to: 

®  attack 

O  avoid  confrontation 


Satellite 


(Not  included  in  sketch) 


Areas  of  Research 

•  Displaying  Levels  of  Abstraction  for  various 

Users 

-  “Simplified  Land  Picture” 

•  Provide  Alternate  Modalities 

-  Integrate  Voice/Chat/Email/Message  Traffic 
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-  ‘Symbicons’ 

-  When  do  symbols/graphs  work  better? 

•  Automatic  Stress/Workload  Detection 

•  Automatic  Display  of  Threats 
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Evaluation 


•  Go  beyond  the  HAT  report  model  - 

-  Because  “cognitive  walkthroughs”  and  comparable  methods  are  NOT 
good  predictors  of  usability  in  practice  (John’s  DARPA  project). 

•  Elaborate  on  what  the  system  is  used  for.  Develop  a  representative 
sample  of  tasks. 

•  Develop/agree  upon  measures  of  task  performance,  process  as  well 
as  ultimate  outcome. 

•  If  the  system  exists  - 

-  Conduct  experimental  studies  of  users  performing  the  sample  tasks 

•  Reasonable  #  of  users  >10 

•  Appropriate  prior  training  before  performance  measurements 

•  If  the  system  does  not  yet  exist ,  but  there  is  a  design  - 

-  Apply  cognitive  modeling  to  the  sample  of  tasks  (Card,  Moran  &  Newell, 
Kieras  &  John) 

•  May  reveal  that  task  requirements  cannot  be  met  (Tomahawk  case) 

•  Can  compare  alternative  designs  (telephone  workstation  case) 

•  Today,  somewhat  limited  scope  of  what  can  be  modeled 
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